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(54) System and method for Indexing between trick play and normal play video streams in a video 
delivery system 

(57) A system and method for indexing between vid- 
eo streams in an interactive video delivery system. The 
interactive video delivery system includes at least one 
media server which stores video streams having differ- 
ent presentation rates. In one embodiment the system 
stores a normal play stream and one or more corre- 
sponding trick play streams. The trick play video 
streams are fast forward and/or fast reverse video 
streams. The system generates index tables or look-up 
tables between the normal play and trick play video 
streams which enable indexing between the streams, 
and uses these look-up tables to switch back and forth 
between the streams. In creating the index tables, the 
system first analyzes the normal play stream and cre- 
ates a normal play time standard based on presentation 
timestamps from the normal play stream. The system 
then creates an index table or look-up table for each of 
the normal play and trick play video streams using the 
normal play time standard. Each index table includes an 
array of two-tuples, wherein the two-tuples are the nor- 
mal play time standard and an index or offset into the 
respective stream. The index tables enable indexing be- 
tween the streams. During video delivery, the system 
uses the respective index tables to switch back and forth 
between the normal play and trick play video streams. 
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Description 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates generally to video delivery and video-on-demand systems, and more particularly to 
a video server system and method for indexing between video streams having different presentation rates. i.e., normal 
play, fast forward and fast reverse video streams. ■ * *' . 

Description of the Related Art 

Video^n-demand or video delivery systems enable a plurality of users or viewers to selectively watch movies or 
other audioVideo sequences which are stored on one or more video servers or media servers. The video servers are 
r connectedjhrough data transfer channels, such as a broadcast cable system or satellite broadcast system, to the 
plurality ofjjsers or subscribers. The video servers store a plurality of movies or other audio/video sequences, and 
x each user can select one or more movies from the video servers for viewing. Each user includes a television or other 
viewing device, as well as associated decoding logic, for selecting and viewing desired movies. When a user selects 
a movie, the selected movie is transferred on one of the data transfer channels to the television of the respective user 
Full-motion digital video requires a large amount of storage and data transfer bandwidth. Thus, video-on-demand 
systems use various types of video compression algorithms to reduce the amount of necessary storage and data 
transfer bandwidth. In general, different video compression methods exist for still graphic images and for full-motion 
. video. Video compression methods for still graphic images or single video frames are referred to as intraframe com- 
pression methods, and compression methods for motion video are referred to as interframe compression methods. 

Examples of video data compression for still graphic images are RLE (Run-Length-Encoding) and JPEG (Joint 
Photographic Experts Group) compression. The RLE compression method operates by testing for duplicated pixels in 
a single line of the bit map and storing the number of consecutive duplicate pixels rather than the data for the pixel 
itself. JPEG compression is a group of related standards that provide either lossless (no image quality degradation) 
or lossy (imperceptible to severe degradation) compression types. Although JPEG compression was originally designed 
for the compression of still images rather than video. JPEG compression is used in some motion video applications. 
In contrast to compression algorithms for still images, most video compression algorithms are designed to com- 
press full motion video. Video compression algorithms for motion video use a concept referred to as interframe com- 
pression, which involves storing only the differences between successive frames in the data file. Interframe compres- 
sion stores the entire image of a key frame or reference frame, generally in a moderately compressed format. Succes- 
sive frames are compared with the key frame, and only the differences between the key frame and the successive 
frames are stored. Periodically, such as when new scenes are displayed, new key frames are stored, and subsequent 
comparisons begin from this new reference point. It is noted that the interframe compression ratio may be kept constant 
while varying the video quality. Alternatively, interframe compression ratios may be content<iependent, /.a, if the video 
clip being compressed includes many abrupt scene transitions from one image to another, the compression is less 
efficient. Examples of video compression which use an interframe compression technique are MPEG, DVI and Indeo. 
among others. 

MPEG Background 

A compression standard referred to as MPEG (Moving Pictures Experts Group) compression is a set of methods 
for compression and decompression of full motion video images which uses the interframe compression technique 
described above. MPEG compression uses both motion compensation and discrete cosine transform (DCT) processes 
and can yield compression ratios of more than 200:1 . 

A general background to and more information about MPEG can be found in the ISO/I EC MPEG specification 
referred to as ISO/lEC 13818, which is hereby incorporated by reference in its entirety. 

The MPEG standard requires that sound be recorded simultaneously with the video data, and the video and audio 
data are interleaved in a single file to attempt to maintain the video and audio synchronized during playback. The audio 
data is typically compressed as well, and the MPEG standard specifies an audio compression method such as MPEG 
Layer II, also known by the Philips trade name of 'MUSICAM'. 

An MPEG stream includes three types of pictures, referred to as the Intra (I) frame, the Predicted (P) frame, and 
the Bi-directional Interpolated (B) frame. The I or Intra frames contain the video data for the entire frame of video and 
are typically placed every 10 to 15 frames. Intra frames provide entry points into the file for random access, and are 
generally only moderately compressed. Predicted frames are encoded with reference to a past frame, i.e., a prior Intra 
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frame or Predicted frame. Thus P frames only include changes relative to prior I or P frames. In general, Predicted 
frames receive a fairly high amount of compression and are used as references for future Predicted frames. Thus, both 
I and P frames are used as references for subsequent frames. Bi-directional pictures include the greatest amount of 
compression and require both a past and a future reference in order to be encoded. Bi-directional frames are not used 
5 for references for other frames. 

After the I frames have been created, the MPEG encoder divides each I frame into a grid of a suitable size, e.g.. 
16 x 16 pixel-squares, called macro blocks. The respective I frame is divided into macro blocks in order to perform 
motion compensation. Each of the subsequent pictures after the I frame are also divided into these same macro blocks. 
The encoder then searches for an exact, or near exact, match between the reference picture macro block and those 
to in succeeding pictures. When a match is found, the encoder transmits a vector movement code or motion vector. The 
vector movement code or motion vector only includes information on the difference between the reference frame and 
the respective succeeding picture. The blocks in succeeding pictures that have no change relative to the block in the 
reference picture or frame are ignored. In general, for the frame(s) following a reference frame, i.e.. P and B frames 
that follow a reference I or P frame, only small portions of these frames are different from the corresponding portions 
is of the respective reference frame. Thus, for these frames, only the differences are captured, compressed and stored. 
Thus the amount of data that is actually stored for these frames is significantly reduced. 

After motion vectors have been generated, the encoder then tracks the changes using spatial redundancy. Thus, 
after finding the changes in location of the macro blocks, the MPEG algorithm further reduces the data by describing 
the difference between corresponding macro blocks; This is accomplished through a math process referred to as the 
20 discrete cosine transform or DCT. This process divides the macro block into a suitable number of sub blocks, e.g.. four 
sub blocks, seeking out changes in color and brightness. Human perception is more sensitive to bnghtness changes 
than color changes. Thus the MPEG algorithm devotes more effort to reducing color space rather than brightness. 

Each picture or frame also includes a picture header which identifies the frame and includes information for that 
frame The MPEG standard also includes sequence headers which identify the start of a video sequence. Sequence 
2S headers are only required once before the beginning of a video sequence. However, the MPEG-2 standard allows a 
sequence header to be transferred before any I frame or P frame. The sequence header includes information relevant 
to the video sequence, including the frame rate and picture size, among other information. 
MPEG video streams used in digital television applications generally include a sequence header before every I frame 
and P frame This is necessary to facilitate channel surfing between different video channels, which is an important 
30 userrequiremenLlngeneral,whenauserswitchestoanewchannel.theyideoforthenewchannelcannotbedis 

until the next sequence header appears in the stream. This is because the sequence header includes important infor- 
mation about the video sequence which is required by the decoder before the sequence can be displayed. If a sequence 
header were not included before each I frame and/or P frame, then when the user switched to a new channel, the 
video for the new channel possibly could not be immediately displayed, i.e., the video could not be displayed until the 
3S next sequence header. 

The sequence headers in an MPEG encoded stream include presentation timestamps or a time base within the 
encoded stream Timestamps provide a user with a time reference relative to the beginning of a movie, enabling the 
user to accurately select or identify a sequence located midstream of the movie without having to reference the begin- 
ning of the movie. 
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Trick Play Streams 



In an interactive videc-on-demahd (VOD) or near-video-on-demand (NVOD) system, it is greatly desirable for the 
user to be able to selectively fast forward and/or fast reverse through the movie being watched. Thus, some videoon- 
« demand systems include fast forward and fast reverse streams, referred to as trick play streams, for each movie. When 
the user desires to fast forward or fast reverse through a movie, the user selects the fast forward or fast reverse option. 
The respective fast forward or fast reverse trick play stream is then transferred to the user at the appropnate point 
•where the user was watching, instead of the normal play stream, thus simulating a fast forward or fast reverse of the 
movie being watched. Typically, a single video stream, such as a movie, is encoded at different presentation rates to 
so enable the video file to operate in fast forward or fast reverse speed in addition to the normal play presentation rate. 



Indexing 



Interactive video-on-demand systems which include trick play streams require methods for indexing between the 
55 normal play stream and the trick play streams, as well as for indexing between the trick play streams In other words, 
when a user is watching a movie and chooses to fast forward for a period of time, a mechanism is needed for the video 
server to switch from the normal play stream to the appropriate point or frame in the fast forward stream. When the 
user then desires to resume watching at normal play speed, a mechanism is also needed for the video server to switch 
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from the frame being viewed in the fast forward stream to the appropriate point or frame in the nan™, , . 

a first video file at a first presentation rate to a second video file at a second presentation rate ■"''"•Putting 
h-J 0ne approach ,or jndexin 9 between n< "mal play and trick play streams includes using look-up tables to inriov 
orlTra^ 

? I " StreamS - F ° r eXamp ' 9 - Wex '^-"P tab,es 030 be generated using the MPEG presemS 
timestamps from the sequence headers of the normal play stream. Presentation 

One drawback to this approach is that the MPEG presentation timestamps may not alwavs be avails r„ 
ample, there is no requirement that the MPEG presentation timestamps be o2aST.?^o3d^^ ^ 
gaps in the presentation timestamps. n»nuous, e.g., mere could be breaks or 

i« c^°, the L Prob 'T * T 3t presentation timestamps are presentation-based. Thus, when a fast forward stream which 
■ 6c fast . being played, the presentation timestamps do not advance 5x faster, but advance mZS^SSl 
do in a normal play stream. Thus in this method the seiver is required to perform computations onTe Jl«„l? X 

option in this method the decoder is required to provkte information back tome ^^5rE2^ 
entatont^estemp where the decoder stopped playing, as well as the presentation rate of the tElSSSSSJ 

rSvlt *™ f " USBS ^ mf0mab0n l ° de,ermine *» Potation timestamp^S beat 

playing in the new stream. This requirement that the decoder interact with the media server to ar™,i£h 9 

Sfm 88 ^ 8 -7 " as , the h compu,ations required 10 be performed » th ° med * ^^SZ^ISZ 

One such approach based on MPEG presentation timestamps is HP's 'PictureNumber PresentationTim^m,, 
RleOffset- format for eaoh table entry. Unfortunate*, not all encoding formats m^SSTSS^Sl 
mappmg between presentation rates can be accomplished only if the underlying assumption that Dresen'taTo^ 2 
s a constant ratio. i.e. one assumes the encoded video stream has a uniform^ frame ^^oSSStSZ 
frame rate at all presentation rates disables techniques such as 'scene fast forward" ^^sely, a uniform 

trfckS^? imPr ° Ved ¥*H T d meth0d iS d88ired ,0f effiCiently W "*W between normal play streams and 
the processing burdens of the media server. reauces 

SUMMARY O F THE INVENTION 

s^z ^r * m T Ct "* VW6 ° SySlem Pre,erab,y comprises 31 ,east media server^ ch 

SH^S^H! 9 ^es, m the preferred embodiment, the system stores a noma, 

play stream and one or more corresponding trick play streams. The trick play video streams are fast forward andb 
fast reverse vrdeo streams. The present invention generates index took-up tables (ILUTs) between me^ll p^ 

fotSK^^ 

system firs, anajzes the norma, play stream and creates a normally time standard based cT eLma bn^meT- 
^s compr^ed m the norma, play stream,The system then preferably creates an index .ook-up^e for 55the 
S^ 3 ^ Each index tabte comprises^ ar^ of 

Tht S'tT J? tW !: tUPl9S ^ 018 n0fmal P ' ay ,ime S,andard ■ nd<n or <** "to toe respective stream 
The index tables enable indexing between the streams. stream. 

During video delivery, the system of the present invention uses the respective index tables to switch back and forth 

%22Z2J£ T\ k 9 me V,de ° Slream ' *" media S6,Ver examines "» cu ™t norma, ptey Le and 
offset of the normal play stream bemg output in order to half the normal play stream at an appropriate point The media 

?ES2.T the T e K nt no T' p, f y ,ime 10 re,rieve appropr * ,e « in *• fast ^^£2? 

S Rafted When the user discontinues fas, forwarding and selects normal play, the media server examines the 
pnate point. The medra server also uses the current normal play time of the fast forward stream to retrieve the appro- 
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priate offset in the normal play stream index table. This offset is then used to begin play of the normal play stream at 
the location where the first forward stream was halted. Similar operations occur when the user fast reverses through 
the video stream. The present invention also provides a smooth transition between streams having different presen- 
tation rates by ensuring that stoppage and initiation of output of the different streams, i.e.. switching the output between 
the different streams, only occurs at well defined "random access" points. 

Therefore, the present invention efficiently allows indexing between normal play and trick play streams. The present 
invention creates a normal play time standard which is used as a common reference, thus simplifying the indexing 
process. This eliminates the requirement of any intelligence in the decoder and reduces the processing requirements 
of the video server. 

DESCRIPTION OF THE DRAWINGS 

A better understanding of the present invention can be obtained when the following detailed description of the 
preferred embodiment is considered in conjunction with the following drawings, in which: 

IS 

Figure 1 illustrates a video delivery system including one or more media servers and one or more subscribers; 
Figure 2 illustrates the media server of Figure 1 ; 

Figure 3 is a block diagram illustrating the media server computer system of Figure 2; 

Figure 4 is a flowchart diagram illustrating generation of index look-up tables for normal play and trick play streams 
20 according to the present invention; 

Figure S illustrates index look-up tables for normal play and trick play streams according to the present invention; 

and 

Figure 6 is a flowchart diagram illustrating operation of the media server indexing between a normal play and trick 
play streams according to the present invention. 

25 

DESCRIPTION OF THE PREFERRED EM BODIMENT 
Video Delivery System 

so Referring now to Figure 1 , a video server or video delivery system 30 fpr storing and transferring video streams 

is shown The system 30 is preferably a video-on-demand (VOD) or near-videoon-demand (NVOD) system, or other 
type of video delivery system, which is capable of transferring or playing video or multimedia streams to one or more 
users preferably a plurality of users. In the present disclosure, the term "video stream" is used to refer to a file or 
sequence of data for presenting a video display. The term Video stream" also includes a multimedia stream which 

3$ includes both video and audio components. 

As shown, in one embodiment the video delivery system 30 comprises one or more media servers or video servers 
50 connected through a broadband network 40 to a plurality of subscribers 52. As discussed below each media server 
50 preferably includes a general purpose computer system 60 (Fig. 2). The broadband network 40 is preferably a 
network suitable for multimedia content, such as an ATM (Asynchronous Transfer Mode) network. The subscribers 52 

40 preferably include display devices such as televisions, computers, etc. 

The media server 50 is capable of transferring or playing video or multimedia streams having different presentation 
rates In the preferred embodiment, the system 50 is capabie of transferring or playing either a normal play stream or 
one or more trick play streams. The trick play streams may comprise one or more of a fast forward and/or fast reverse 
stream Thus, in the present disclosure, the term trick play streams" refers to fast forward and/or fast reverse video 

45 streams, preferably compressed streams, which are generated from a normal play stream, and which have a different 
presentation rate than the normal play stream. - - • ^ , ...... . 

As noted above, the normal play and trick play streams are preferably compressed video streams. An embodiment 
of the invention operates independently of the type or format of the video streams. Thus the video streams may be 
compressed in any of various types of formats, including MPEG-1, MPEG-2. Motion JPEG. QuickTime, etc. Further. 

so an embodiment of the invention operates independently of the frame rate and other presentation characteristics. 

Figure 2 - Media Server 

Referring now to Figure 2. in this embodiment the media server or video server 50 comprises a computer system 
ss 60 Figure 3 is a block diagram illustrating the components comprised in the media server computer system 60 of 
Figure 2 The media server computer system 60 includes various standard components, including one or more proc- 
essors, one or more buses, a hard drive and memory. It is noted that Figure 3 is illustrative only, and other computer 
architectures may be used, as desired. As shown, the computer system 60 includes at least one processor 80 coupled 
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through chipset logic 82 to a system memory 84. The chipset 82 includes a PCI fPerioheral Comonn*n» 
bndgafor««e rf acln 9 toPC. bus86. The computer system 60 in^^ 

disk array 90 or other storage media tor storing the normal ptey streams and corresponding trick S2 
computer system 60 may include either or both of an MPEG decoder 74 and MPEcTenco^rre Sch IT,^ 
connected to PC. bus 86. The computer system 60 may also include video circuit^ as^ ^ 
Referring again to Figure 2. the computer system 60 includes or is coupled to one or more digital storaoe or 

62 through cable 64. The media storage unit 62 may be in addition to. or instead of a disk storaoe JsiZ in T 
computer system 60. The media storage unit 62 includes one or more composite fSm^t^S^^ 
play streams i and corresponding trick play streams. Alternately, the media storage unit eS^hWhSZ^J 
or more Cf>ROMdrivesand/oroneorrnore Digital Video Disk (DVD)storage units o^ 

d.g.tal video. The computer system 60 may also include one or 'more i^2^^r^^^t22 
dig^^ 

The compressed normal play and trick play streams may be comprised on a storaoe media in th» martt * » 

TlZrl T m T *° S,0ra9e m6dia pr0Vides the ** out to the one or more display units o 
Snth ' r *, m T ™" 50 OUtput me video usi "9 various communication nUHuc a ATM 
£222?" T ,er M0d8) '. ,SDN ( ' nte9rated Sen,iCe8 Di9rtal Netwo ' k >- or via As noted above Te fZ 

STeS? COmPnSe te,eViSi0nS ' SyS,emS ° f 0th6r SyS,BmS 3 «:re e nYor d is %% 

n^n^.S 0 ^ ab ° Ve ' T 6d ! a 86,Ver 50 ind6XeS ° r 8Witches be,ween normal P ,av a™* trick play video streams 
generally based on user selections. As discussed further below, the media server 50 generates Lex tebles S 
various streams and uses these tables to switch between the various streams. In M LboS^TSSSil toSe 
ll^T^ ^ 0 ,u , nc,tons afe P""™* by the media server 50 in software, wherein tZso^re is re P S 

I^TJ 2 - !" ana,h-r embodiment - *• ^ 60 includes dedicated hardware whfchSer- 

forms one or both of the index table generation and indexing functions. P 

th.t !I " me media setver 50 mav emprise two or more interconnected computers, as desired It is noted 

that any of vanous types of video ***y systems may be used according to the preset menion lstesl^ 

Figure 4 - Creation of Index Look-up Tables 

' in J^^T™ l ?, Fi9Ur8 4 ' 3 dia9fam il,ustratin 9 generation of index look-up tables (LUTs) according to the present 

system. The drfferent streams preferably encode the same content for presentation at different rates 

As shown in step 102, server 50 receives or examines a normal play video stream or multimedia stream As 

££L Se9m6nt ° r m ° Vi6, ° n, ° 3 SCreea such as a ,elevisk,n or a com P ut *' ^tem. in this JESSS 

^e normal ptey stream . « compressed stream, preferably an MPEG* compressed stream, although other typToi 
may be used, as desired. According* the index LUTs are generated using the -*Bho MPE^SESS 

«t Jl 8t , e h P 1M ; 86rVer ? ° 8031X268 times,am P s witnin ,he In this embodiment where the stream is an MPEG 

a £r,h V T , ana ^ eS thS presenta,ion ^stamps from the sequence headers in the stream. As mentioned 
above, the presentation timestamps are used to provide a time base for the video sequence menuonea 

a JUS^T?^' a " MP _ EG ° ncoded s,ream inc,udes a Plenty of I frames which are intracoded pictures and 
VSEF h ? P ^ ^ afe in,erCOded ,rames: The 1 frames each ***** vWe ° *ta for an enie t rame 
o 22? m ^r^" 7 h *• SeqUenCa ^ P and 8 ,rames delude change information reS to prbr 
or subsequent frames. Each picture or frame also includes a picture header which identifies the frame and includes 
.nforma t,on for that frame An MPEG encoded stream further includes one or more sequence headers vSt ude 

Z?!£ °" T"?" 8 *" Vide ° S6qUenCe ' inC,Udin9 ** ,ram8 fate and *« pict " re 5ize - ««8 other Worma 
lion The sequence headers include presentation timestamps which indicate the play time of the video sequence 

in step 106 server 50 maps the presentation timestamps to a 'normal play time' (NPT) standard. Thus server 50 

Z Tposil" IT? r" d d °" 1,19 ^ °' NPT *" <*" be aSS0Cia,ed «* « ■ p ° sition " -thin a murme Ja 
Wla Posmons are defined to be equivalent between multimedia or video streams having different presentation rates 

when the content present at the respective position is conceptually substantially equivalent. Hence, for video data the 
SSS 1 ? !5 <0 equivalent ^ en t" 6 same or substantially the same image in the sequence is being presented 
allowing for differences in resolution and other encoding parameters than may be particular to the stream 

In an embodiment of the invention, NPT provides an indication of contextual position within a compressed video 
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stream, or any other multimedia file, by assigning an increasing numeric value to succeeding elements, e.g.. f rahnes 
or sequences, in the stream. As a result. NPT permits the location of a particular item of content within the video stream 
to be determined regardless of the presentation rate, encoding scheme or storage medium. 

In generating the normal play time standard, server 50 examines the presentation timestamps and keeps track of 

s the original or base presentation timestamp at the beginning of the movie. Server 50 then subtracts that base or original 
presentation timestamp from subsequent timestamps to determine the normal play time values for the normal play 
stream, thereby compensating for any non-zero base presentation timestamp. In other words, in order to calculate 
normal play time for a given point in the normal play stream, the system subtracts the base presentation timestamp 
from a future presentation timestamp at the respective point of location in the normal play stream to determine the 

10 normal play time value for that location. 

In this embodiment, the normal play time (NPT) for a position in a multimedia stream is the time from the beginning 
of the title until the respective position when measured by presentation of the normal speed forward or normal play 
stream. Therefore, the concept of normal play time is used. Normal play time corresponds to the speed of the normal 
play stream and has a one to one correspondence with clock time. Thus, every second the normal play movie ticks 

is forward, normal play time clicks forward one second. In a fast forward file or fast forward trick play stream, if the FF 
stream is 5x faster, normal play time is 5x faster as the user is watching. 

In general, any particular scene in the movie is identified by a normal play time. Thus, if a particular scene occurs 
at X minutes into the movie in normal play time, then this position or scene is referred to as or called X minutes. This 
particular scene is also located in any of the other trick play streams at X minutes normal play time. Thus in the fast 

20 forward and fast reverse streams, even though time is going by much faster, at X minutes normal play time the particular 
scene occurs. 

In step 108 server 50 creates an index look-up tables for each ol the multimedia streams, i.e.. for the normal play 
stream and each of the trick play streams. The index look-up table for the normal play multimedia stream comprises 
an index or array of two-tuples. The index look-up tables for a normal play, fast forward, and fast reverse stream are 
25 shown in Figure 5. As shown, each tuple comprises a normal play time value and a corresponding file offset within the 
stream. 

Note that the entries in each NPT index may be constrained by requirements of the encoding scheme. For example, 
some encoding schemes may only allow random positioning into the encoded stream at certain non-linear intervals. 
In the case of an MPEG2 transport stream, the "random access indicator' is set withfri the transport packet header to 
so indicate the file offsets of the respective encoded data packets and resulting NPT indices are •randomly accessible". 

For the normal play stream, the normal play time entries comprise the normal play time values computed in step 
106. For the scaled streams, e.g., the fast forward and fast reverse streams, a scale factor is introduced into the normal 
play time values of the index look-up tables to compensate for the different presentation rates. Scaling of the presen- 
tation timestamps can be accomplished by multiplying the compressed presentation timestamp value by the ratio of 
3S the presentation rate to the normal presentation rate. 

It is noted that equivalent positions in multimedia streams having different presentation rates will have equal NPT 
values although the actual time presentation from the beginning of the stream to that position will differ for the different 
streams It is also noted that equivalent positions in multimedia streams having different presentation rates, although 
having equal NPT values, will have different byte offsets due to a presumptive difference in length of the streams having 
*» different presentation rates. 

The index look-up tables specify indices or entries each based on a normal play time and a file offset to allow the 
multimedia server 50 to initiate or stop play at a particular normal play time point in the multimedia stream. The index 
look-up table indices also allow the multimedia server 50 to transfer to and between equivalent positions between 
streams of different presentation rate. i.e., between normal play and trick play streams. The index look-up table only 
4S includes tuples representing valid positions for starting stopping, or transferring between the streams. 

The creation of the look-up tables is independent of any particular type of video compression or MPEG represen- 
tation Hence where MPEG compression is used the index look-up tables are created by scanning through the MPEG 
file noting random access points in the MPEG file to compensate for presentation timestamp discontinuities, and then 
converting from the presentation timestamp in the MPEG file into the normal play time standard. Conceptually, each 
so index table comprises an array of normal play time vs. scenes, and any particular image in the movie can be identified 
by the normal play time value. As noted above, an index table is created for each presentation rate, e.g., fast forward, 
fast reverse, and normal play. Each of the offsets stored in the index table is an index from the normal play time to a 
byte offset in that MPEG file where the particular scene begins. 

Hence server 50 uses a normal play time standard, instead of using timestamps in the video stream. As noted 
ss above thepresentation timestamps in an MPEG sequence are typically not always available. For example, there is 
no requirement that the timestamps be continuous, e.g.. there could be breaks or gaps in the presentation timestamps. 
Therefore, unlike prior art methods, an embodiment of the invention does not use the presentation timestamps as a 
basis for creating the index tables. Instead, an embodiment of the invention maps the presentation timestamps to a 
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normal play time standard, and this normal play time standard is then used as a basis for creating the index tables. 
Figure 6 - Transferring Between Streams 

* Referring now to Figure 6..a flowchart diagram illustrating operation of the system of the present invention ,»„ 
femng ouyute between multimedia streams having different presentation rates* shown .Her TlZSS Z a 
current v.deo or mutaedia stream is being output from the media server 50. and the media serveTsTh^™, ? !• 
user mput indicating that a different stream should be output' For example ihTm^^ m« h« « ? ! 
-, no ^P'aymulUmediastream.andthemediaserver50rece^^ 

When user input is received indicating a desired change in the presentation rate then in stee £23- 
50 finds a tuple to the index tab.e of me current stream oMto^L^^cS^^ 
In other words, assuming that the current stream is playing and is at a certain point or offset within Z LTf^ , 
.202 the rnedfc server 50 finds the tuple or entry h the Index table of the cJZZ^^^^ 

software executing in the media sewer 50. In step 202 the media server 50 receives mis b^S ' 

- r e rourre:«~ 

202 T„^;^!T e ^ 

neL^iuh^ , tS Tf * 8 CUrfem S,ream bei " 9 ""P* me media sewer 50 P^^ly finds the tuple to Z 
nearestsubsequentnormalplaytimeor nearest offsetof the locate 

offset of this type to terminate play of the current video stream at this offset associated 
. In step 206 the media server 50 determines the normal play time for the current stream ft is noted that the n« ma . 
Ptey time for the current stream may have been previously determined in step 202. In other woXk^o deZS 
the nearest offset greater than the byte offset of the current output stream in step 202 the correVn^Sn^T 
t.me value in this tuple may be used as the normal P .ay time fo°the curiam C ° rreSp ° nd,n9 normal 
In step 208 media server 50 finds the tuple in the index table of the new stream. i.e . the stream to be outa.it with 

: 2£2? ?2T "l 8,ep 208 10 initiate ou,put of 1,19 new 8,ream at ■* «** Th « ^put of tta!?JZX 

Therefore, initiation and termination of the output of a respective stream being output at a given normal olav tima 
a^Pl'shed by finding the tuple in the respective index table for the nearest normal ptey Sme 
associated fi.e offset as the point to initiate or terminate pfcy of the stream. TransfernnXwee,^ fd^SSJE 

T^zz^ 9 m :r presenta,ion ra,es is » «**g ^ in 9 £c^ 

iTbeg^^^^^ 

6 tr JltT^ 9 ^ 60 ' in ?!l ,i0n PTOVideS a System me,hod ,or M,xin 9 ^tween normal play and trick play video 
streams. An embodiment the invention examines the presentation timestamps in me sequence heade s SnoS 

£? Tf^ CreatSS 3 n0rma ' P ' ay Ume Standard "*"* is used tor a » breams. The system then creltesS 

s*ndi^ 

8pond,ng offse s mto the respective stream. During play, the video delivery system uses these index tables to He fi- 

° f ! X b6tWeen n0rma ' Play and ,riCk P ' ay Streams - ^ Vproach also permits iSSSSi pres- 
entation rates such as scene forward or presentation rates based on content complexity. non " con8 »'» P™' 

rfpn^ r » m0d i Cat,0n8 b9 U8Sd ,0 9enera,a a normal pla y ,in ? e standard tor «he normal play stream without 
SSSJL k PrSS f nt inV6nti0n - F ° r eXamp,e ' NPT index LUTs <*" also be generated prioMo enccSl cTthe 
Video streams by using frame numbers or sequence numbers,Alternatively. the NPT indices may be generated I con 
currently with the video cedent encoding.^ 

by the frame rate. This NPT position can then be associated with the file offset of the encoded frame 

* tJZlZT* pre f senta,ions < Le - positiv9 Presentation rate scale factor), the NPT value for each file offset at which 
a Random Access Point occurs may be calculated using the equation: 

• NPT = (PresentationRateScaleFactor • FileBitOffset) / ConstantBitRate 
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Conversely, for reverse presentations (i.e. negative presentation rate scale factor), the NPT value for each file 
offset may be calculated using the equation: 

NPT = TotalNPT + ((PresentationRateScaieFactor * FileBitOffset) / ConstantBitFlate) 

Wherein: 

PresentationRateScaieFactor = ratio of presentation rate with respect to normal presentation rate (e.g. a value of 
10 7 indicates 7x fast forward, a value of -5 indicates 5x fast reverse): 

FileBitOffset = the number of bits from the beginning of the encoding to the file offset specified as a random access 
point; 

ConstantBitRate = the constant bit rate at which the encoding is intended to be played; and 
TotalNPT = the total time duration of normal speed presentation (Le. what would be commonly though of as the 
is length of the movie). 

Although the system and method of the present invention has been described in connection with the described 
embodiments, it is not intended to be limited to the specific form set forth herein, but on the contrary, it is intended to 
cover such alternatives, modifications, and equivalents, as can be reasonably included within the scope of the invention. 

20 

Claims 

1 . A computer-implemented method for indexing a first and a second related video stream having different presen- 
ts * tation rates, the method comprising the step of creating index look-up tables (LUTs) for each of said related video 

streams using a normal play time standard, wherein each of said index LUTs includes a plurality of entries com- 
prising a normal play time value and a corresponding offset into the respective video stream. 

2. The method of claim 1 . wherein the video streams include a normal play stream, wherein said creating index look- 
so up tables for said first and second video streams includes: 

receiving the normal play stream, wherein the normal play stream includes a plurality of timestamps; and 
mapping said plurality of timestamps to said normal play time standard; 

3S wherein said creating index look-up tables for said first and second video streams uses said normal play 

time standard. 

3. The method of claim 2, wherein said video streams having different presentation rates comprise MPEG compressed 

streams; s 
40 wherein said mapping said plurality of timestamps to a normal play time standard comprises examining 

sequence headers in said MPEG compressed normal play stream for said plurality of timestamps. 

4. The method of claim 1 . wherein said video streams-include trickplay streams including a fast forward stream and 
a fast reverse stream. . 

45 

5. The method of claim V.* wherein said normal play time has a one to one correspondence to clock time. 

. 6. A computer-implemented method for transitioning between a first and a second related video stream having dif- 
ferent presentation rates, the method comprising: 

so 

creating index look-up tables (LUTs) for each of said related video streams using a normal play time standard, 
wherein each of said index LUTs includes a plurality of entries comprising a normal play time value and a 
corresponding offset into the respective stream; 
transferring video data from said first video stream; and 
55 switching between said first video stream and said second video stream using said index look-up tables. 

7. The method of claim 6. wherein the video streams include a normal play stream, wherein said creating index look- 
up tables for said first and second video streams includes: 
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receiving the normal play stream, wherein the normal play stream includes a plurality of timestamps- and 
mapping sad plurality of timestamps to said normal play time standard: . Bwam »*- and 

time sS."" Cfea,in9 ind6X ,0 ° k * UP ^ '° r fifSt lnd SeC ° nd S,reams uses sai < norma. play 

8. The method of claim 7. wherein said video streams comprise MPEG compressed streams- 

• wheren said mapping said plurality of timestamps to a normal play time standard' comprises examin,™ 
sequence headers in said MPEG compressed normal play stream for said plurality of SeS ps 9 

9. The method of claim 6. wherein said switching between said first and second streams includes: 

determining the normal play time of said first video stream; 

determining an offset in the second video stream based on the normal play time of the first video stream- a „n 
.nitafng output of the second vfcieo stream at said determined offset in the seind I vi«eo sUSm 

determining the offset in said found entry in said index look-up table of said second video stream. 

* * 

11. The method of claim 9. wherein said switching between said first and second video streams further comprises: 
cte^i^^ 

o^rx^ 

12 ' ^l^fit^t ? er8in Vid8 ° S,feamS havi " 9 di " erent Potation rates include trickplay streams 
including a fast forward stream and a fast reverse stream. s«eams 

13. The method of claim 6, wherein said normal play time has a one to one correspondence to clock time. 

14. A computer-implemented method for transitioning between a first and a second related video stream each said 
video stream having different presentation rates, the method comprising: 

transferring video data from said first video stream; 

referencingan index look-up table (LUT) for each said videostream. wherein each said LUT includes a plurality 
gentries composing a normal play time (NPT) value and a corresponding offset into the respective video 

switching from said first video stream to said second stream using the index LUTs; and 
transferring video data from said second video stream. 

15. The method of claim 14. wherein the video streams include a normal play stream, wherein said creatino ind«* 
look-up tables for said first and second video streams includes: Crea, ' n9 ,ndex . 

receiving the normal play stream, wherein the normal play stream includes a plurality of timestamps- and 
mapping said plurality of timestamps :o said normal play time standard: ' 

time standard ^ ,W ^ a " d SeCOnd video s,reams uses said «w«nal'play 

1 6. The method of claim 1 5. wherein said video streams comprise MPEG compressed streams- 

Mu^lT T W ■ maPP Jl 9 . J P,Urality °' ,imestam P s to a normal «me standard comprises examining 
sequence headers in sa.d MPEG compressed normal play stream for said plurality of timestamps 
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17. The method of claim 1 4, wherein said switching between said first and second streams includes: 

determining the normal play time of said first video stream; 

determining an offset in the second video stream based on the normal play time of the first video stream; and 
s initiating output of the second video stream at said determined offset in the second video stream. 

18. The method of claim 17. wherein said determining an offset in the second video stream based on the normal play 
time of the first video stream comprises: 

jo finding an entry in an index look-up table of said second video stream having a normal play time value close 

to the normal play time of the first video stream; and 

determining the offset in said found entry in said index look-up table of said second video stream. 

19. The method of claim 17, wherein said switching between said first and second video streams further comprises: 

determining an entry in an index table for the first video stream that contains an offset beyond an offset of data 
currently being output from the second video stream; and 

scheduling output of the current stream to terminate at said offset beyond the offset of data currently being 
output from the first video stream. 

20. The method of claim 14 t wherein the video streams having different presentation rates include trickplay streams 
including a fast forward stream and a fast reverse stream. 

21 . The method of claim 1 4, wherein said normal play time has a one to one correspondence to clock time. 

22. A video server which provides video streams having different presentation rates, wherein the video server indexes 
" between said video streams having different presentation rates, the video server comprising: 

video memory configured to store the video streams having different presentation rates; 
an index look-up table (LUT) for each of said video streams, wherein the index look-up tables are based on 
a normal play time standard, wherein each of said index look-up tables includes a plurality of entries comprising 
a normal play time value and a corresponding offset into the respective stream; 

one or more output ports coupled to said video memory for transferring video data from a video stream; and 
a switch coupled to said video memory and said memory, and configured to switch between said video streams 
3$ at said one or more output ports, wherein said switch uses said index look-up tables in switching between said 

video streams. 

23 The video server of claim 22,wherein the video streams include a normal play stream, the video server further 
configured to create indices for said LUTs by examining the normal play stream which includes a plurality of times- 

40 tamps and by mapping said plurality of timestamps to said normal play time standard. 

24 The video server of claim 22. wherein said video streams includes MPEG compressed streams, and said LUT 
generator maps said plurality of timestamps to said normal play time standard by examining sequence headers 
in said MPEG compressed normal play stream for said plurality of timestamps. 
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25 The video server of claim 22,f urther configured to determine the normal play time of a current stream being played. 

to determine an offset in a new stream based on the normal play time of the current stream, and to initiate output 
. of the new stream at said determined offset in the new stream. 

bo 26 The video server of claim 25. further configured to locate an entry in an index look-up table of said new stream 
having a normal play time value close to the normal play time of the current stream, and to determine the offset 
in said found entry in said index look-up table of said new stream. 

27 The video server of claim 25. further configured to determine an entry in an index table for the current stream that 
55 ' contains an offset beyond an offset of data currently being output from the current stream and to schedule output 
of the current stream to terminate at said offset beyond the offset of data currently being output from the current 
stream. 
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28. The video server of claim 22. wherein the video streams having different presentation rate* hri,* 
streams including a fast forward stream and a fast reverse stream Dresentat, ° n «*» "elude tnckplay 

29. The video server of claim 22. wherein said normal play time has a one to one correspondence to clock time. 
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25 



an Max look-up table creation program configured to create index look-up tables for each of said vid eo «tro am 
us.ng a normal play time standard, wherein each of said index ^r^k^ J!^J^ 
compns.ng a norma, play time value and a corresponding offset into the respite stream 

a mapping program configured to map said plurality of timestamps to said normal play time standard; 
wherein said index look-up table creation program uses said normal play time standard. 
32. The computer-readable storage media of claim 30. wherein said video stream switching program includes: 
a program configured to determine the normal play time of the current stream- 

a program configured to initiate.output of the new stream at said determined offset in the new stream. 

M ' ^L^T'^T^ St °[ a f m6dia °' C ' aim *• Wher8in ^ P f ?9 ram ,or determining an offset in the new 
^ stream based on the normal play time of the current stream comprises: 

iSf 1 < f , ? UMd ,0 f "? *" enUy an index l00k ' up teb,e of ** new 8,rea ™ having a normal play time 
value close to the normal play time of the current stream: and 

a program configured to determine the offset in said found entry in said index look-up table of said new stream. 
* M - ™«°"^ter-readables^^^ 

a program configured to determine an entry in an index table for the current stream that contains an offset 
beyond an offset of data currently being output from the current stream- and 

a program configured to schedule output of the current stream to terminate at said offset beyond the offset of 
data currently being output from the current stream. ««yona me onset or 

' ' 3S ' 1"!?! SUbscri P ,ion : living and displaying video streams having different presentation rates from a 

• VidS ° SSrVer indeXSS b6,Ween SaW Video ^ams'having different present ^s 

the video subscription system comprising: 1 

I ^nf Play 1^ 6 con " 9 ." red !° dis P la y « ne video s,f eams having the different presentation rates; and 
awntrallerow^ 

2 1 te °hS r c e Z S h y h'" 9 ^ '?° k " UP tab ' e (LUT) for each °' said video s,reams - -e index 

look-up teb es are based on a normal play tone standard, wherein each of said index look-up tables includes 
a plurality of entr.es comprising a normal play time (NPT) value and a corresponding offset into the respective 
stream, and wherein sad VI deo streams include a normal play stream and said entries are created by exam- 
• ning the normal play stream which includes a plurality of timestamps and by mapping said plurality of times- 
tamps to said normal play time standard. K-u.auiyoi umes 



45 



SO 



ss 



BNSOOCID:<£P «S12112A2J_> 



12 




BNSOOaD: <EP_C8121 \ZA2Jj> 



13 



EP 0 812 112 A2 




BNSDOCID:<£P 06121 12A*J_> 



14 



EP 0 812 112 A2 



80 

A 



84 



82 



CPU 
(PROCESSING) 



CHIPSET 



SYSTEM 
MEMORY 



86 

A 




88 



4 



PCI BUS 



MPEG 
DECODER 




MPEG 
ENCODER 




DISK 
ARRAY 


I 

74 


1 

76 


I 

90 



FIG. 3 



BNSOOaO: <£P 06121 12A2_L> 



15 



EP 0812 112 A2 



RECEIVE 
NORMAL PLAY 
MPEG STREAM 



102 



EXAMINE SEQUENCE 
HEADERS FOR 
PRESENTATION 
TIMESTAMPS 



MAP PRESENTATION 

TIMESTAMPS TO 
NORMAL PLAY TIME 



104 



106 



CREATE INDEX LOOK-UP 
TABLES FOR NORMAL 

PLAY. FAST FORWARD. 
AND FAST REVERSE 
STREAMS 



108 



FIG. 4 



BNSOOaO: <£P 06121 12A2J_> 



16 



EP0812 112 A2 



u. 

O u 

is 

UJ 



li! ^ 



EE 
o 



UJ 
CO 

en ~ 
UJ uj w 

2 o o- »- 
z 



O 

X 

o 

IU 



ai 
S 



Q 

sli 

2 



*4 CO 
j= ti- 
ll, u- 

o 

•J 

^ £ UJ 

251 

z 




uj -! S u 
^ < o: -j 

5 g22 



z 
o 




17 



BNSOOat): <EP_06121 12A2J_> 



EP 0812 112 A2 



TRANSFER BETWEEN 
MULTIMEDIA STREAMS 



FIND TUPLE IN THE INDEX 
TABLE FOR THE CURRENT 
STREAM THAT CONTAINS AN 

OFFSET BEYOND THE 
CURRENT OUTPUT OFFSET OF 
THE CURRENT STREAM 



202 



SCHEDULE THE CURRENT 
STREAM TO TERMINATE 

OUTPUT AT THIS 
DETERMINED OFFSET 



204 



DETERMINE THE NORMAL 
PLAYTIME FOR THE 
CURRENT STREAM 



206 



FIND TUPLE IN THE INDEX 
TABLE FOR THE NEW STREAM 
WITH THE NEAREST NORMAL 
PLAY TIME TO THE NORMAL 
PLAY TIME OF THE CURRENT 
STREAM 



208 



INITIATE OUTPUT OF THE NEW 

STREAM AT THE OFFSET OF 
THE FOUND TUPLE AFTER THE 
CURRENT STREAM 
TERMINATES 



210 



FIG. 6 



BNSOOQO: <£P_06121 \2A2Jj> 



18 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 



□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 



□ FADED TEXT OR DRAWING 




BLURRED OR ILLEGIBLE TEXT OR DRAWING 



<ousn))|NOT839VdsiHl 



